Reinforcement Learning in Self Organizing Cellular Networks

Self-organization is a key feature as cellular networks densify and become more heterogeneous, through the additional small cells such as pico and femtocells. Self- organizing networks (SONs) can perform self-configuration, self-optimization, and self-healing. These operations can cover basic tasks such as the configuration of a newly installed base station, resource management, and fault management in the network. In other words, SONs attempt to minimize human intervention where they use measurements from the network to minimize the cost of installation, configuration, and maintenance of the network. In fact, SONs aim to bring two main factors in play: intelligence and autonomous adaptability. One of the main requirements for achieving such goals is to learn from sensory data and signal measurements in networks. Therefore, machine learning techniques can play a major role in processing underutilized sensory data to enhance the performance of SONs. In the first part of this dissertation, we focus on reinforcement learning as a viable approach for learning from signal measurements. We develop a general framework in heterogeneous cellular networks agnostic to the learning approach. We design multiple reward functions and study different effects of the reward function, Markov state model, learning rate, and cooperation methods on the performance of reinforcement learning in cellular networks. Further, we look into the optimality of reinforcement learning solutions and provide insights into how to achieve optimal solutions. In the second part of the dissertation, we propose a novel architecture based on spatial indexing for system-evaluation of heterogeneous 5G cellular networks. We develop an open-source platform based on the proposed architecture that can be used to study large scale directional cellular networks. The proposed platform is used for generating training data sets of accurate signal-to-interference-plus-noise-ratio (SINR) values in millimeter-wave communications for machine learning purposes. Then, with taking advantage of the developed platform, we look into dense millimeter-wave networks as one of the key technologies in 5G cellular networks. We focus on topology management of millimeter-wave backhaul networks and study and provide multiple insights on the evaluation and selection of proper performance metrics in dense millimeter-wave networks. Finally, we finish this part by proposing a self-organizing solution to achieve k-connectivity via reinforcement learning in the topology management of wireless networks

Multi-Stream LDPC Decoder on GPU of Mobile Devices

Low-density parity check (LDPC) codes have been extensively applied in mobile communication systems due to their excellent error correcting capabilities. However, their broad adoption has been hindered by the high complexity of the LDPC decoder. Although to date, dedicated hardware has been used to implement low latency LDPC decoders, recent advancements in the architecture of mobile processors have made it possible to develop software solutions. In this paper, we propose a multi-stream LDPC decoder designed for a mobile device. The proposed decoder uses graphics processing unit (GPU) of a mobile device to achieve efficient real-time decoding. The proposed solution is implemented on an NVIDIA Tegra board as a system on a chip (SoC), where our results indicate that we can control the load on the central processing units through the multi-stream structure

Self-Organizing mmWave Networks: A Power Allocation Scheme Based on Machine Learning

Millimeter-wave (mmWave) communication is anticipated to provide significant throughout gains in urban scenarios. To this end, network densification is a necessity to meet the high traffic volume generated by smart phones, tablets, and sensory devices while overcoming large pathloss and high blockages at mmWaves frequencies. These denser networks are created with users deploying small mmWave base stations (BSs) in a plug-and-play fashion. Although, this deployment method provides the required density, the amorphous deployment of BSs needs distributed management. To address this difficulty, we propose a self-organizing method to allocate power to mmWave BSs in an ultra dense network. The proposed method consists of two parts: clustering using fast local clustering and power allocation via Q-learning. The important features of the proposed method are its scalability and self-organizing capabilities, which are both important features of 5G. Our simulations demonstrate that the introduced method, provides required quality of service (QoS) for all the users independent of the size of the network

Joint Power Allocation in Interference-Limited Networks via Distributed Coordinated Learning

Dense deployment of small base stations (SBSs) is one of the main methods to meet the 5G data rate requirements. However, high density of independent SBSs will increase the interference within the network. To circumvent this interference, there is a need to develop self-organizing methods to manage the resources of the network. In this paper, we present a distributed power allocation algorithm based on multi-agent Q-learning in an interference-limited network. The proposed method leverages coordination through simple message passing between SBSs to achieve an optimal joint power allocation. Simulation results show the optimality of the proposed method for a two-user case

Lens-Based Millimeter Wave Reconfigurable Antenna NOMA

This paper proposes a new multiple access technique based on the millimeter wave lens-based reconfigurable antenna systems. In particular, to support a large number of groups of users with different angles of departures (AoDs), we integrate recently proposed reconfigurable antenna multiple access (RAMA) into non-orthogonal multiple access (NOMA). The proposed technique, named reconfigurable antenna NOMA (RA-NOMA), divides the users with respect to their AoDs and channel gains. Users with different AoDs and comparable channel gains are served via RAMA while users with the same AoDs but different channel gains are served via NOMA. This technique results in the independence of the number of radio frequency chains from the number of NOMA groups. Further, we derive the feasibility conditions and show that the power allocation for RA-NOMA is a convex problem. We then derive the maximum achievable sum-rate of RA-NOMA. Simulation results show that RA-NOMA outperforms conventional orthogonal multiple access (OMA) as well as the combination of RAMA with the OMA techniques

Reinforcement Learning for Self Organization and Power Control of Two-Tier Heterogeneous Networks

Self-organizing networks (SONs) can help manage the severe interference in dense heterogeneous networks (HetNets). Given their need to automatically configure power and other settings, machine learning is a promising tool for data-driven decision making in SONs. In this paper, a HetNet is modeled as a dense two-tier network with conventional macrocells overlaid with denser small cells (e.g. femto or pico cells). First, a distributed framework based on multi-agent Markov decision process is proposed that models the power optimization problem in the network. Second, we present a systematic approach for designing a reward function based on the optimization problem. Third, we introduce Q-learning based distributed power allocation algorithm (Q-DPA) as a self-organizing mechanism that enables ongoing transmit power adaptation as new small cells are added to the network. Further, the sample complexity of the Q-DPA algorithm to achieve ϵ-optimality with high probability is provided. We demonstrate, at density of several thousands femtocells per km2, the required quality of service of a macrocell user can be maintained via the proper selection of independent or cooperative learning and appropriate Markov state models

Geometric morphometric analysis in nine species of genus Hottentotta (Birula 1908) (Arachnida: Scorpiones) from Iran

Hottentotta Birula, 1908 is one of the most widely distributed buthid scorpions, with more than 40described species from Africa, across the Middle East, to India. Currently, this genus is representedby ten morphological species in Iran (H. akbarii, H. jayakari, H. juliae, H. khoozestanus, H.lorestanus, H. navidpouri, H. saulcyi, H. schach, H. sistanensis and H. zagrosensis), all of which areendemic or subendemic in Iran. The members of this genus have not been properly studied from thetaxonomic point of view. A tool that could contribute to scorpions' taxonomic studies is geometricmorphometry, which is defined as the fusion between geometry and biology. In this study, the sizeand shape variations in sternocoxal structure in Hottentotta populations have been examined usingthe geometric morphometric method. The goal was to analyze the isometric size and conformation innine species of Hottentotta. 100 individuals of Hottentotta, collected from different parts of Iranduring 2018-2020, were photographed. Coordinate (x, y) configurations from landmarks wereregistered in sternocoxal structures. Geometric morphometric analyses were performed using Rlanguage. The results clearly showed divergence in the shape and size of sternocoxal structureamong the studied taxa. However, the major shape changes were associated with H. akbarii whichhas a larger size of sternocoxal structure and a narrower sternum, shorter coxa II-III, and longercoxa IV

The Effect of Gates-Glidden Drills on the Quality of Root Canal Treatment by Pre-Clinical Dental Students

AIM: This study was conducted to investigate the effect of applying Gates-Glidden (GG) drill by pre-clinical dental students on root canal treatment quality. METHOD: A total of 56 first molars consisting of 168 canals were selected in this study. For this purpose, 56 students who had been formerly trained by two methods of root canal preparation were randomly divided into two groups (n = 28). Group 1: the step-down method by GG and Group 2: step-back technique without GG. The prepared teeth were filled with gutta-percha/ZOE sealer using lateral condensation. Periapical radiographs were taken before and the following treatment to survey occurrence of preparation errors and CBCT images to determine residual dentine at furcation region. RESULTS: The findings showed that among 10 error types in specimens prepared by students, the occurrence of underfilling, overfilling, inappropriate, ledge formation, and single cone was more common without GG. There were no significant differences in residual dentine amount at furcation region between preparation with and without using GG (P > 0.05). CONCLUSION: Using GG for root canal preparation by dental students resulted in low errors and not an increased dentine removal risk

Comparison of Depression, Anxiety, Stress and Quality of Life in Drug Abusers with Normal Subjects

AbstractObjective: This study compared depression, anxiety, stress and quality of life in drug abusers with normal population. Given that the above mentioned factors are pivotal in continuing addiction. Materials and Methods: In this comparative study, one hundred drug abusers who were admitted to quit addiction clinic in Rasht with one hundred normal people who were relatives of patients or staff in health centers as control group underwent study. Depression, anxiety and stress were assessed by DASS-21 and sf-36 questionnaire was used for quality of life assessment. Analysis of the results was performed using SPSS software (ver. 16).Results: The results showed that compared with normal individuals addicted to opiates significantly depression, anxiety and stress were higher. The quality of life of ordinary people was also significantly higher than those addicted to opiates. Depression, anxiety and stress were found to be negatively correlated with quality of life. Conclusion: Based on our findings, we can say, addiction, depression, anxiety and stress are related to the formation of a vicious cycle where addicts due to the loss of prestige and hit a by stander family, and the feelings of guilt and the legal treatment of depression, anxiety and more stress than individuals with and taking refuge in the lap of addiction try to get rid of these thoughts and feelings. This leads to a vicious cycle which will eventually lead to low quality of life for these individuals

Sperm matrix metalloproteinase-2 activity increased in pregnant couples treated with intrauterine insemination: a prospective case control study

Matrix metalloproteinase-2 (MMP2) and matrix metalloproteinase-9 (MMP9) have an important role in the reproductive system and in the fertilisation process. The aim of this study was to investigate the MMP2 and MMP9 activity in semen and their association with the pregnancy rate, semen parameters and seminal plasma oxidative stress parameters in couples who were treated with intrauterine insemination (IUI). The semen specimens were obtained from 60 men who attended with their spouse for the IUI in the infertility unit. A controlled ovarian stimulation was performed with clomiphene citrate in IUI cycles. Women with positive pregnancies were recorded (n = 29). The results showed the activity of sperm MMP2 and seminal plasma MMP9 was significantly higher in the pregnant group, compared to the non-pregnant group (p < .05). There was a correlation between the sperm MMP2 activity and the total thiol group (TTG) (r = 0.276, p < .05) and the total antioxidant capacity (TAC) of seminal plasma (r = 0.304, p < .05). The sperm MMP9 showed a positive correlation with the seminal plasma TAC (r = 0.330, p < .05) and an inverse correlation with the lipid peroxidation (LP) of seminal plasma (r = –304, p< 0.05). In addition, the seminal plasma MMP2 activity was correlated to sperm viability (r = 0.266, p< .05) and the TTG of seminal plasma (r = 0.298, p < .05). The MMP2 activity in the sperm may be an important factor for determining the pregnancy rate after IUI.Impact statement What is already known on this subject? Previous studies have reported that the fusion between the sperm and zona pellucida required the activity of matrix metalloproteinase 2 (MMP2), whereas the inhibition of MMP2 can significantly decrease the in vitro fertilisation (IVF) rate. What do the results of this study add? This study has identified that the sperm MMP2 activity was significantly higher in the pregnant couples in comparison with the non-pregnant couples, who treated with intrauterine insemination (IUI). The findings showed there was a correlation between sperm MMP2 activity and the total thiol group (TTG) and the total antioxidant capacity (TAC) of the seminal plasma. What are the implications of these findings for clinical practice and/or further research? MMP2 activity in the sperm could influence the IUI outcome and it is an important factor for IUI success